Comparing Learned Representations between Unpruned and Pruned Deep Convolutional Neural Networks

While deep neural networks have shown impressive performance in computer vision tasks, natural language processing, and other domains, the sizes and inference times of these models can often prevent them from being used on resource-constrained systems. Furthermore, as these networks grow larger in size and complexity, it can become even harder to understand the learned representations of the input data that these networks form through training. These issues of growing network size, increasing complexity and runtime, and ambiguity in the understanding of internal representations serve as guiding points for this work. In this thesis, we create a neural network that is capable of predicting up to three path waypoints given an input image. This network will be used in conjunction with other networks to help guide an autonomous robotic vehicle. Since this neural network will be deployed to an embedded system, it is important that our network is efficient. As such, we use a network compression technique known as L1 norm pruning to reduce the size of the network and speed up the inference time, while retaining similar loss. Furthermore, we investigate the effects that pruning has on the internal learned representations of models by comparing unpruned and pruned network layers using projection weighted canonical correlation analysis (PWCCA). Our results show that for deep convolutional neural networks (CNN), PWCCA similarity scores between early convolutional layers start low and then gradually increase towards the final layers of the network, with some peaks in the intermediate layers. We also show that for our deep CNN, linear layers at the end of the network also exhibit very high similarity, serving to guide the dissimilar representations from intermediate convolutional layers to a common representation that yields similar network performance between unpruned and pruned networks

Session 2: Female Orgasms and Evolutionary Theory

Proceedings of the Pittsburgh Workshop in History and Philosophy of Biology, Center for Philosophy of Science, University of Pittsburgh, March 23-24 2001 Session 2: Female Orgasms and Evolutionary Theor

The Craft of Incentive Prize Design: Lessons from the Public Sector

In the last five years, incentive prizes have transformed from an exotic open innovation tool to a proven innovation strategy for the public, private and philanthropic sectors. This report offers practical lessons for public sector leaders and their counterparts in the philanthropic and private sectors to help understand what types of outcomes incentive prizes help to achieve, what design elements prize designers use to create these challenges and how to make smart design choices to achieve a particular outcome. It synthesizes insights from expert interviews and analysis of more than 400 prize

DANNA2: Dynamic Adaptive Neural Network Arrays

Traditional Von Neumann architectures have been at the center of computing for decades thanks in part to Moore\u27s Law and Dennard Scaling. However, MOSFET scaling is rapidly approaching its physical limits spelling the end of an era. This is causing researchers to examine alternative solutions. Neuromorphic computing is a paradigm shift which may offer increased capabilities and efficiency by borrowing concepts from biology and incorporating them into an alternative computing platform.The TENNLab group explores these architectures and the associated challenges. The group currently has a mature hardware platform referred to as Dynamic Adaptive Neural Network Arrays (DANNA). DANNA is a digital discrete spiking neural network architecture with software, FPGA, and VLSI implementations. This work introduces a successor architecture built on the lessons learned from prior models. The DANNA2 model offers an order of magnitude improvement over DANNA in both simulation speed and hardware clock frequency while expanding functionality and improving effective density

Modifications on Translation Initiation

Two studies by Meyer et al. and Wang et al. demonstrate a role for m6A modification of mRNA in stimulating translation initiation. These findings add to the growing number of diverse mechanisms for translation initiation in eukaryotes

Structure/activity relationships applied to the hydrogenation of α,β-unsaturated carbonyls: The hydrogenation of 3-butyne-2-one over alumina-supported palladium catalysts

The gas phase hydrogenation of 3-butyne-2-one, an alkynic ketone, over two alumina-supported palladium catalysts is investigated using infrared spectroscopy in a batch reactor at 373 K. The mean particle size of the palladium crystallites of the two catalysts are comparable (2.4 ± 0.1 nm). One catalyst (Pd(NO3)2/Al2O3) is prepared from a palladium(II) nitrate precursor, whereas the other catalyst (PdCl2/Al2O3) is prepared using palladium(II) chloride as the Pd precursor compound. A three-stage sequential process is observed with the Pd(NO3)2/Al2O3 catalyst facilitating complete reduction all the way through to 2-butanol. However, hydrogenation stops at 2-butanone with the PdCl2/Al2O3 catalyst. The inability of the PdCl2/Al2O3 catalyst to reduce 2-butanone is attributed to the inaccessibility of edge sites on this catalyst, which are blocked by chlorine retention originating from the catalyst’s preparative process. The reaction profiles observed for the hydrogenation of this alkynic ketone are consistent with the site-selective chemistry recently reported for the hydrogenation of crotonaldehyde, an alkenic aldehyde, over the same two catalysts. Thus, it is suggested that a previously postulated structure/activity relationship may be generic for the hydrogenation of α,β-unsaturated carbonyl compounds over supported Pd catalysts

Inelastic neutron scattering studies of methyl chloride synthesis over alumina

Not only is alumina the most widely used catalyst support material in the world, it is also an important catalyst in its own right. One major chemical process that uses alumina in this respect is the industrial production of methyl chloride. This is a large scale process (650 000 metric tons in 2010 in the United States), and a key feedstock in the production of silicones that are widely used as household sealants. In this Account, we show how, in partnership with conventional spectroscopic and reaction testing methods, inelastic neutron scattering (INS) spectroscopy can provide additional insight into the active sites present on the catalyst, as well as the intermediates present on the catalyst surface.<p></p> INS spectroscopy is a form of vibrational spectroscopy, where the spectral features are dominated by modes involving hydrogen. Because of this, most materials including alumina are largely transparent to neutrons. Advantageously, in this technique, the entire “mid-infrared”, 0–4000 cm<sup>–1</sup>, range is accessible; there is no cut-off at 1400 cm<sup>–1</sup> as in infrared spectroscopy. It is also straightforward to distinguish fundamental modes from overtones and combinations. <p></p> A key parameter in the catalyst’s activity is the surface acidity. In infrared spectroscopy of adsorbed pyridine, the shifts in the ring stretching modes are dependent on the strength of the acid site. However, there is a very limited spectral range available. We discuss how we can observe the low energy ring deformation modes of adsorbed pyridine by INS spectroscopy. These modes can undergo shifts that are as large as those seen with infrared inspectroscopy, potentially enabling finer discrimination between acid sites. <p></p> Surface hydroxyls play a key role in alumina catalysis, but in infrared spectroscopy, the presence of electrical anharmonicity complicates the interpretation of the O–H stretch region. In addition, the deformations lie below the infrared cut-off. Both of these limitations are irrelevant to INS spectroscopy, and all the modes are readily observable. When we add HCl to the catalyst surface, the acid causes changes in the spectra. We can then deduce both that the surface chlorination leads to enhanced Lewis acidity and that the hydroxyl group must be threefold coordinated. <p></p> When we react η-alumina with methanol, the catalyst forms a chemisorbed methoxy species. Infrared spectroscopy clearly shows its presence but also indicates the possible coexistence of a second species. Because of INS spectroscopy’s ability to discriminate between fundamental modes and combinations, we were able to unambiguously show that there is a single intermediate present on the surface of the active catalyst. This work represents a clear example where an understanding of the chemistry at the molecular level can help rationalize improvements in a large scale industrial process with both financial and environmental benefits. <p></p&gt

Psychometric evaluation of disordered eating measures in bariatric surgery candidates

Introduction: Assessment of disordered eating is common in bariatric surgery candidates, yet psychometric properties of disordered eating measures in this population are largely unknown. Methods: Measures were completed by 405 adult bariatric surgery candidates at pre-surgical consultation. Fit of the original scale structures was tested using confirmatory factor analysis (CFA) and alternative factor solutions were generated using exploratory factor analysis (EFA). Reliability (internal consistency), construct validity (convergent and divergent) and criterion validity (with the EDE as criterion) were assessed. Materials: The measures prioritised for evaluation are the following: Eating Disorder Examination Questionnaire (EDE-Q; n = 405), Three-Factor EatingQuestionnaire (TFEQ; n = 405), Questionnaire of Eating and Weight Patterns Revised (QEWP-R; n = 204), Clinical Impairment Assessment (CIA; n = 204) and the Eating Disorder Examination clinical interview (EDE; n = 131). Results: CFA revealed adequate fit for only the CIA in its current form (CFI = 0.925, RMSEA = 0.096). EFA produced revised scales with improved reliability for the EDE, EDE-Q and TFEQ. Reliability of revised subscales was improved (original scales α = 0.43–0.82; revised scales α = 0.67–0.93). Correlational analyses of the CIA and revised versions of remaining scales with measures of psychological wellbeing and impairment revealed adequate convergent validity. All measures differentiated an EDE-classified disordered eating group from a non-disordered eating group (criterion validity). Diagnostic concordance between the EDE, EDE-Q and QEWP-R was low, and identification of disordered eating behaviours was inconsistent across measures. Conclusions: Findings highlight the limitations of existing disordered eating questionnaires in bariatric surgery candidates. Results suggest revised assessments are required to overcome these limitations and ensure that measures informing clinical recommendations regarding patient care are reliable and valid

Model Cards for Model Reporting

Trained machine learning models are increasingly used to perform high-impact tasks in areas such as law enforcement, medicine, education, and employment. In order to clarify the intended use cases of machine learning models and minimize their usage in contexts for which they are not well suited, we recommend that released models be accompanied by documentation detailing their performance characteristics. In this paper, we propose a framework that we call model cards, to encourage such transparent model reporting. Model cards are short documents accompanying trained machine learning models that provide benchmarked evaluation in a variety of conditions, such as across different cultural, demographic, or phenotypic groups (e.g., race, geographic location, sex, Fitzpatrick skin type) and intersectional groups (e.g., age and race, or sex and Fitzpatrick skin type) that are relevant to the intended application domains. Model cards also disclose the context in which models are intended to be used, details of the performance evaluation procedures, and other relevant information. While we focus primarily on human-centered machine learning models in the application fields of computer vision and natural language processing, this framework can be used to document any trained machine learning model. To solidify the concept, we provide cards for two supervised models: One trained to detect smiling faces in images, and one trained to detect toxic comments in text. We propose model cards as a step towards the responsible democratization of machine learning and related AI technology, increasing transparency into how well AI technology works. We hope this work encourages those releasing trained machine learning models to accompany model releases with similar detailed evaluation numbers and other relevant documentation